Papers with multimodal foundation model
Fact-Aware Multimodal Retrieval Augmentation for Accurate Medical Radiology Report Generation (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing multimodal foundation models suffer from serious factual inaccuracy in radiology report generation. |
| Approach: | They propose a fact-aware multimodal retrieval-augmented pipeline for generating accurate radiology reports using RadGraph. |
| Outcome: | The proposed multimodal retrieval-augmented pipeline outperforms state-of-the-art retrievers on language generation and radiology-specific metrics. |
VIP5: Towards Multimodal Foundation Models for Recommendation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in foundation models have impeded the ability for these fields to benefit from each other’s advancements. |
| Approach: | They propose to use a multimodal foundation model to unify various modalities and recommendation tasks under the P5 recommendation paradigm to implement personalized prompts. |
| Outcome: | The proposed model will unify visual, textual, and personalization modalities under the P5 recommendation paradigm and will improve recommendation performance and efficiency. |